Reinforcement Learning for Argumentation: Describing a PhD Research

Artificial intelligence (AI) is increasingly studied in many fields such as philosophy, law and decision making. One of the approaches to AI is the use of agent and multi-agent systems. Agents are key element for building complex largescale distributed systems[9]. In multi-agent systems, each agent interacts with the environment and communicates with other agents in order to achieve the designated goal. Communication means to share and exchange information, cooperate and coordinate with each other in order to achieve a common goal. Argumentation is a type of communication between agents and a process attempting to form an agreement about what to believe. There has been increasing research in argumentation and dialogue systems in the past decade[23]. The agent as a dialogue participant needs sophisticated dialogue strategies in order to make high quality dialogue contributions. By reviewing the state of art literature in computerised dialogue systems (e.g.[21];[22]), it is observed that their dialogue strategies (i.e. strategic heuristics) are hardwired into the computational agent. One of the main issues with this is that an agent might be incapable of dealing with new dialogue situations that have not been coded, and indeed this is an impossible task given the dynamic nature of argumentation. It would be ideal to make an agent search for an optimal strategy by itself e.g. via trial and error, and thus the agent with the best strategy will win the argument [8]. Machine learning has an important role to play in order to meet these challenges. To make agents learn the dialogue strategies, it would be more flexible for them to make an argument through exploration (trial and error). It is believed that learning can make agents more flexible to adapt to new environments and new dialogue situations. One of the popular machine learning approaches with regards to learning agents is known as reinforcement learning (RL). Reinforcement learning focuses on how to map an action for each state by interacting with the environment and observing the state change[15]. Sutton and Barto[15] define reinforcement learning as an agent learning what to do and how to connect each situation with an action to maximise the cumulative reward. The learner or agent is not told what action should be taken, rather the learner needs to explore a policy that yields the maximum cumulative reward by trying them out. In reinforcement learning, the agent interacts with the environment by taking an action and receiving a reward for the action taken as seen in figure 1. To make an